REPORT  DOCUMENTATION  PAGE 


Form  Approved 
0MB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  Is  estimated  to  average  1  hour  per  response,  including  the  time  for  revieiwing  instructions,  searching  data  sources. 

gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  Information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection 

of  information,  Including  suggestions  for  reducing  this  burden  to  Washington  Headquarters  Service,  Directorate  for  Information  Operations  and  Reports, 

1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget, 

Paperwork  Reduction  Project  (0704-0188)  Washington,  DC  20503. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS.  _ _ 


2.  REPORT  DATE 


1 .  REPORT  DATE  (DD-MM-YYYY) 

01-09-1999 


4.  TITLE  AND  SUBTITLE 


Annual  Performance  Report 


VISUAL  GAIT  MODIFICATION  1®^  YEAR  TECHNICAL  REPORT 


6.  AUTHOR(S) 


LEWIS,  M.  ANTHONY 
ETIENNE-CUMMINGS,  RALPH 


3.  DATES  COVERED  (From  -  To) 

1  JULY  1999-30  SEPT  1999 


5a.  CONTRACT  NUMBER 

N/A 


5b.  GRANT  NUMBER 

N0001 4-99-1 -0984 


5c.  PROGRAM  ELEMENT  NUMBER 

N/A 


5d.  PROJECT  NUMBER 

N/A 


5e.  TASK  NUMBER 

N/A 


5f.  WORK  UNIT  NUMBER 

N/A 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

THE  JOHNS  HOPKINS  UNIVERSITY 
HOMEWOOD  RESEARCH  ADMINISTRATION 
105  AMES  HALL  4300  NORTH  CHARLES  ST 
BALTIMORE,  MD  21218-2686 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

OFFICE  OF  NAVAL  RESEARCH 
ONR  252:  DIANE  GALES 
BALLSONT  CENTRE  TOWER  ONE 
800  NORTH  QUINCY  STREET 
ARLINGTON.  VA  22217-5660 


12.  DISTRIBUTION  AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

ONR 

11.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 

N/A 


14.  ABSTRACT 


THIS  REPORT  DESCRIBES  FIRST  YEAR  ACTIVITIES  IN  DEVELOPING  A 
NEURAL  MODEL  OF  VISUALLY  GUIDED  STEPPING  HOSTED  IN  A  WALKING  MACHINE. 
WORK  IN  THE  DEVELOPMENT  OF  A  NEURAL  CPG  CHIP,  AS  WELL  AS  A 
A  MODEL  OF  VISUALLY  TRIGGERED  GAIT  MODIFICATION  IS  DESCRIBED. 


1$.  SECURmr  CLASSIFICATION  OF:  ^ 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

17.  UMITATIONOF 
ABSTRACT 


18.  NUMBER  19a.  NAME  OF  RESPONSIBLE  PERSON 
OF  PAGES 

19b.  TELEPONE  NUMBER  (Inctude  area  code) 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANS(-Std  Z39-18 


Dnc  QUALmr  iwspbctbd  4  looonont 


Visual  Gait  Modification  Year  Technical  Report 

Grant  No.  NOOO 14-99-0984 
Dr.  Thomas  McKenna,  Program  Officer 


July  1, 1999-September  30,  1999 


Prepared  by 


M.  Anthony  Lewis  Ralph  Etienne-Cummings 

Iguana  Robotics,  Inc  The  Johns  Hopkins  University 


Abstract-  This  report  describes  first  year  activities  in  developing  a 
neural  model  of  visually  guided  stepping  hosted  in  a  walking  machine. 
Work  in  the  development  of  a  neural  CPG  chip,  as  well  as  a  a  model  of 
visually  triggered  gait  modification  is  described. 


1.  introduction 


The  collaboration  between  The  Johns  Hopkins  University  and  Iguana  Robotics,  Inc. 
has  gotten  off  to  a  good  start.  In  early  July  the  Pis  tested  an  aVLSI  CPG  chip  by 
controlling  a  robotic  leg  running  on  a  rotating  drum.  We  were  able  to  establish  the  basic 
functionality  of  the  chip  and  show  that  it  could  be  used  in  closed  loop  to  control  a  simple 
leg.  The  results  of  this  experiment  are  described  in  Section  2. 

Secondly,  we  revised  and  prepared  for  publication  an  article  for  a  special  issue  of 
Connection  Science  on  the  topic  of  Adaptive  Robots.  This  article  details  a  model  of 
visually  triggered  gait  modification.  This  article  has  been  accepted  for  publication  and 
should  appear  later  this  year  in  a  special  issue  on  adaptive  robots. 

Thirdly,  administrative  issues  have  been  addressed  and  the  subcontract  agreement 
between  The  Johns  Hopkins  University  and  Iguana  Robotics,  Inc.  has  been  completed. 

Finally,  at  a  meeting  held  at  The  Johns  Hopkins  University  on  August  22"^  and  23"^, 
the  Pis  outlined  plans  for  work  until  the  end  of  this  calendar  year.  The  major  points 
discussed  are  shown  in  Section  5. 

Also  worth  mentioning  is  that  our  current  work  is  drawing  interest  from  researchers 
and  neuromodelers.  Avis  Cohen  at  the  University  of  Maryland  is  interested  in  the 
possibility  of  applying  the  neural  chip  technology  in  the  restoration  of  function  to 
paraplegics.  The  Pis  have  met  with  Professor  Cohen  several  times  since  July  to  discuss 
the  collaboration.  Also  of  interest,  Nicolas  Schweighofer,  a  research  scientist  in  Kawato’s 
lab  at  ATR  in  Japan  has  offered  to  adapt  a  model  of  cerebellar  learning  for  use  in  the  task 
of  visually  guided  walking.  We  are  optimistic  that  this  collaboration  will  allow  us  test  an 
established  model  of  the  cerebellum  in  the  task  of  robot  control. 

2.  Control  of  a  robotic  leg  using  a  CPG 

This  project  focused  around  the  evaluation  of  a  CPG  chip  previously  designed  and 
fabricated  by  the  Pis.  This  project  was  carried  out  during  the  Telluride  Neuromorphic 
Engineering  Workshop.  This  work  was  done  in  collaboration  with  Avis  Cohen, 

University  of  Maryland,  and  Mitra  Hartmann,  from  the  Bower  lab  at  Caltech. 


2.1.  Objective 

We  achieved  preliminary  results  in  the  control  of  a  robotic  leg  using  a  custom  aVLSI 
neural  circuit.  The  long  range  intention  is  to  integrate  this  chip  with  other  chips  to 
achieve  the  goal  of  visually  triggered  gait  modification.  Currently  we  are  interested  in 
evaluated  the  strengths  and  weaknesses  of  the  current  design. 

The  objective  of  this  particular  experiment  was  to: 

(1)  Demonstrate  an  adaptive  CPG  modeled  as  in  aVLSI. 

(2)  To  control  the  hip  joint  of  a  robot  leg  using  the  output  of  the  CPG  chip. 

(3)  Use  sensory  feedback,  emulating  stretch  receptors,  from  the  hip  joint  to  control  the 
resetting  of  the  CPG. 

2.2.  Physical  Step-Up 

Spiking  Low  Pass 
Oscillator  Neurons  Filters 


Figure  1  On  chip  functions  of  the  CPG  chip. 

In  Fig  1  we  see  a  diagram  of  the  functions  provided  on  the  CPG  chip.  Two  neurons  are 
connected  with  mutually  inhibitory  connections.  These  connections  enforce  a  basic 
coordination  between  the  CPGs.  These  are  non-spiking,  graded  response  neurons  with 
hard  threshold  non-linearites.  These  neurons  can  be  reset  by  external  signals. 

These  neurons  are  then  used  to  drive  spiking  or  bursting  neurons.  These  neurons 
would  be  analogous  to  motor  neurons.  In  the  normal  mode,  bursts  of  activity  alternate 
from  these  two  neurons. 
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Figure  2.  System  diagram.  A  coupled  pair  of  CPGs  generate  extensor 
and  flexor  commands.  These  commands  are  integrated  in  software 
and  the  resulting  position  command  is  used  to  drive  the  hip  joint  of 
the  leg.  Sensor  output  from  the  hip  joint  is  used  to  reset  the  CPGs. 

These  neurons  were  then  interfaced  to  a  servo  motor  using  a  rudimentary  muscle 
model.  A  diagram  showing  the  connection  to  a  robot  leg  is  shown  in  Fig  2. 

The  leg  also  included  sensors  on  the  passive  knee  joint  as  well  as  a  foot  pressure 
sensor.  These  were  used  to  measure  the  performance  of  the  system.  The  only  sensory 
feedback  is  from  the  muscle  as  is  used  to  reset  the  CPG. 

2.3.  Results 

An  experiment  was  performed  demonstrating  the  ability  of  the  stretch  reflex  to 
stabilize  the  running.  Some  of  the  results  are  shown  in  Fig  3.  Here  we  see  that  the  stretch 


reflex  significantly  stabilizes  the  gait  of  the  robot. 
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Figure  3.  Leg  control  results.  (A)  We  see  the  effect  of  lesions  the 
sensory  feedback.  The  hip  position  of  the  leg  drifts  downward 
(indicating  a  backward  drift)  when  sensory  feedback  is  leasioned.  (B) 
This  graph  shows  that  during  lesions,  the  Hip  and  knee  position 
versus  time.  In  the  regions  colored  in  red,  magenta  and  blue,  we  see 
that  the  trajectories  appear  very  regular.  In  dark  blue,  black,  and 
green,  we  see  that  the  trajectories  degenerate  when  the  sensory 
feedback  is  lessioned. 

We  noted  that  in  this  particular  chip,  there  was  difficulty  in  adjusting  the  phase 
difference  between  the  graded  response  neurons  comprising  the  CPG. 


Using  this  simple  experimental  setup  we  were  able  to  evaluate  the  performance  of  the 
CPG  chip.  We  demonstrated  the  ability  to  generate  burst  of  spiking  output,  to  drive  a  leg 
and  to  be  controlled  using  sensory  feedback. 

An  interesting  result  was  that  the  knee  could  be  driven  passively  by  the  dynamics  of 
the  interaction  of  the  ground  and  forced  transmitted  from  the  upper  limb.  The  graph  in 
Fig  3  (B)  shows  the  remarkably  regular  pattern  between  the  knee  and  hip  even  though  the 
knee  joint  is  purely  a  passive  joint. 

Certain  issues  were  identified  in  this  chip.  They  center  on  the  issue  of  parameter 
sensitivity.  We  found  that  certain  parameters  (such  as  the  phase  difference  between  the 
CPG)  were  much  too  sensitive  to  small  variations  in  input  parameters.  As  a  result,  it  was 
impossible,  for  practical  purposes,  to  adjust  certain  parameters  of  the  CPG  bursting.  This 
problem  will  be  addressed  in  the  next  iteration  of  the  chip. 

Other  features  of  the  chip  include  the  ability  to  adjust  the  burst  length  (the  duration  of 
the  spikes  going  to  either  extensor  or  flexor),  the  frequency  of  bursting  (or  the  inter-spike 
interval). 

3.  Publications  and  Presentation. 

The  following  paper  has  been  accepted  for  publication  in  the  Journal  Connection 
Science: 

“Elegant  Stepping:  A  model  of  visually  triggered  gait  modification.”  M. 

Anthony  Lewis  and  Lucia  S.  Simo.  This  paper  will  be  published  later  this 
year. 

Avis  Cohen  will  present  the  work  described  in  Section  2  at  the  2nd  European 
Workshop  on  Neuromorphic  Systems  (EWNS2)  3-5  September  1999  held  in  Sterling, 
Scotland. 

4.  Administrative 

1.  A  subcontract  agreement  between  Iguana  Robotics,  Inc.  and  The  Johns  Hopkins 
University  was  negotiated  and  signed  on  August  16*,  1999. 

2.  An  account  was  setup  at  JHU  for  Etienne-Cummings 


3.  An  organizational  meeting  was  held  at  JHU  on  August  23^^^  -24*  to  discuss  the 

project  organization  and  outline  plans  for  the  rest  of  the  calendar  year. 

5.  Plans  for  October  1®*-Dec  31®* 

The  objective  for  this  quarter  is  to  build  create  advance  the  current  model  of  the 
visually  triggered  gait  modification  develop  the  hardware  infrastructure  needed  to 
perform  systematic  experiments  in  a  real  system.  Specifically: 

1.  A  computer  interface  between  existing  neuromorphic  chips  and  a  computer  system 
will  be  developed.  This  will  allow  the  real-time  transfer  of  data  to  a  computer.  The 
computer  will  then  use  this  information  in  a  real-time  simulation. 

2.  A  new  CPG  chip  will  be  fabricated. 

3.  The  simulation  will  be  extended  to  include  a  more  biologically  realistic  cerebellar 
model  as  well  as  using  realistic  synthetic  visual  data. 

4.  A  leg  of  Geo  III  will  be  designed  and  fabricated. 

5.  A  robot  treadmill  suitable  for  a  small  robot,  will  be  purchased  or  developed.  This 
will  be  a  small  treadmill  that  will  allow  a  robot  to  walk  on  a  continuous  surface. 

6.  A  miniature  vergence  (dual  pan-tilt)  device  will  be  fabricated.  This  will  be  used  as 
part  of  a  system  to  measure  distances  to  obstacles. 


